Existing benchmarks for evaluating long-context language models (LCLMs) primarily focus on long-context recall, requiring models to produce short responses based on a few critical snippets while processing thousands of irrelevant tokens. We introduce LongProc (Long Procedural Generation), a new benchmark that requires both the integration of highly dispersed information and long-form generation. LongProc consists of six diverse procedural generation tasks, such as extracting structured information from HTML pages into a TSV format and executing complex search procedures to create travel plans. These tasks challenge LCLMs by testing their ability to follow detailed procedural instructions, synthesize and reason over dispersed information, and generate structured, long-form outputs (up to 8K tokens). Furthermore, as these tasks adhere to deterministic procedures and yield structured outputs, they enable reliable rule-based evaluation. We evaluate 17 LCLMs on LongProc across three difficulty levels, with maximum numbers of output tokens set at 500, 2K, and 8K. Notably, while all tested models claim a context window size above 32K tokens, open-weight models typically falter on 2K-token tasks, and closed-source models like GPT-4o show significant degradation on 8K-token tasks. Further analysis reveals that LCLMs struggle to maintain long-range coherence in long-form generations. These findings highlight critical limitations in current LCLMs and suggest substantial room for improvement. Data and code available at: https://princeton-pli.github.io/LongProc

Russian President Vladimir Putin has apologised to his Azerbaijani counterpart Ilham Aliyev for what he called a "tragic incident" following the deadly crash of an Azerbaijan Airlines plane this week in Kazakhstan. The plane was flying on Wednesday from Azerbaijan's capital of Baku to Grozny, the regional capital of the Russian republic of Chechnya, when it turned towards Kazakhstan and crashed while attempting to land. In a statement on Saturday, the Kremlin said Russian air defence systems were firing near Grozny due to a Ukrainian drone strike, but stopped short of saying one of these hit the plane. "Vladimir Putin apologised for the tragic incident that occurred in Russian airspace and once again expressed his deep and sincere condolences to the families of the victims and wished a speedy recovery to the injured," the Kremlin said. "At that time, Grozny, Mozdok and Vladikavkaz were being attacked by Ukrainian unmanned aerial vehicles, and Russian air defence systems repelled these attacks."

The Kremlin released a statement on Saturday noting Putin had spoken to Azerbaijan's president Ilham Aliyev by phone. "(President) Vladimir Putin apologised for the tragic incident that occurred in Russian airspace and once again expressed his deep and sincere condolences to the families of the victims and wished a speedy recovery to the injured," the statement said. Prior to Saturday, the Kremlin had not yet commented on the crash. But Russian aviation authorities had said the situation in the region was "very complicated" due to Ukrainian drone strikes on Chechnya. Aviation experts and others in Azerbaijan believe the plane's GPS systems were affected by electronic jamming and it was then damaged by shrapnel from Russian air defence missile blasts.

Russia's Foreign Ministry accused NATO of trying to turn Moldova into a logistical centre to supply the Ukrainian army and of seeking to bring the Western alliance's military infrastructure closer to Russia. Arto Pahkin, the head of operations of the Finnish electricity grid, told the country's public broadcaster Yle that "the possibility of sabotage cannot be ruled out" after an undersea power cable linking Finland and Estonia broke down. It is the latest in a series of incidents involving telecom cables and energy pipelines in the Baltic Sea. A "terrorist act" sank the Russian cargo ship that went down in international waters in the Mediterranean this week, the Russian state-owned company that owns the vessel said. The Oboronlogistika company said it "thinks a targeted terrorist attack was committed on December 23, 2024, against the Ursa Major", without indicating who may have been behind the act or why. The Azerbaijan Airlines passenger jet that crashed near the city of Aktau in Kazakhstan, killing 38 people, was earlier diverting from an area of Russia that Moscow has recently defended against Ukrainian drone attacks.